[pull] main from open-telemetry:main #47

pull · 2024-10-18T10:09:34Z

See Commits and Changes for more details.

Can you help keep this open source service alive? 💖 Please sponsor : )

#### Description Remove the Elasticsearch exporter's `dedup` config. This config setting has been deprecated for 6 months and can be safely removed now. #### Link to tracking issue Follow up to #33773 #### Testing N/A. The config was already a no-op, this just removes the config setting. #### Documentation Removed mention of the config from README.

#### Description The Config struct has a Validate method but it is only used in tests. Ensure we validate the configuration when creating the processor.

…37189) Currently, a "not sampled" decision for a trace is only cached when late spans arrive for it. Whereas a "sampled" decision for a trace is immediately cached. This pull request immediately caches "not sampled" decisions, late spans (or reprocessed spans) for "not sampled" traces now take fewer resources to process. Currently, traces are stored in memory until `num_traces` is reached, then the oldest is deleted to make room. This is a clever mechanism to limit the size* of the internal map, however, trace span counts and their contents likely vary considerably. This pull request doesn't touch this mechanism, but it does delete traces from the internal map after they are released and are added to a decision cache. --------- Signed-off-by: Sean Porter <[email protected]>

Signed-off-by: Bogdan Drutu <[email protected]>

#### Description Fix spelling in exporter/ #37128 (review) #### Link to tracking issue * #37128 --------- Signed-off-by: Josh Soref <[email protected]>

#### Description Fix flaky TestExporterLogs test  #### Link to tracking issue Fixes #37286  #### Testing  #### Documentation

#### Description This MR does the following: * Re adds the ability to allow multiple workers in this exporter due to: 1. Out of Order is no longer an issue now that it is fully supported in Prometheus. Nonetheless, I am setting the default worker as 1 to avoid OoO in Vanilla Prometheus Settings. 2. With a single worker, and for a collector with a large load, this becomes "blocking". Example: Imagine a scenario in which a collector is collecting lots of targets, and with a slow prometheus/unstable network, a single worker can easily bottleneck the off-shipping if retries are enabled. #### Link to tracking issue N/A #### Testing #### #### Documentation docs auto-updated. Readme.md is now correct in its explanation of the `num_consumers since its no longer hard-coded at 1. Additional docs added.

#36419) …e name logic  #### Description Datadog associates two name fields with each span: "resource name" and "operation name." Resource name is analogous to `span.name` in OTel, but operation name has no 1:1 correspondence with any OTel fields. (Read more about what operation name respresents here: https://docs.datadoghq.com/tracing/guide/configuring-primary-operation/) Previously, the datadog exporter [would compute operation name as `{scope name}.{span kind}`](https://github.com/DataDog/datadog-agent/blob/8b92185df483f39456c41766b5666a2bd22307e6/pkg/trace/traceutil/otel_util.go#L367). This would result in long, difficult to read operation names in the Datadog UI - [a prior discussion on this issue can be found here](#1909). In this PR, add a feature gate `datadog.EnableOperationAndResourceNameV2`. This gate enables new logic for operation and resource name that improves readability of these fields by producing shorter names and better aligning with the OTel spec. The names computed in the new logic follow those computed in [dd-trace-go](https://github.com/DataDog/dd-trace-go).  #### Testing Unit tests and E2E tests for the new logic are done in the `datadog-agent` repo. Added a test in this repo that verifies that setting the feature gate enables the new logic; also ran the collector with the feature flag and verified new logic manually.  #### Documentation Changelog added

Proposing to take the position of codeowner of jmxreceiver. Moving this component out of unmaintained status.

…ols (#37295)  #### Description Deprecates githubgen in favor of the new location of this code under opentelemetry-go-build-tools  #### Link to tracking issue Related to #37294

#### Description `githubgen` was adding trailing whitespaces to unmaintained components that was unnecessary. This removes the trailing whitespaces.  #### Link to tracking issue Fixes #37292

#### Description Implements the common queueing feature into the `awss3exporter`, as suggested [here](#36264).

Add JMX metrics gatherer version `1.43.0-alpha`. cc @open-telemetry/java-contrib-approvers

…37298) Doing in place expansion leads to hitting bash line size limits, this is a cheap improvement that makes less likely to hit cases like #37287. Tests on my fork https://github.com/pjanotti/opentelemetry-service-contrib/actions/runs/12836050460/job/35796880868

#### Description This PR adds the vcenter.vm.memory.granted and vcenter.host.cpu.capacity metric. More information on this metric can be found [here](https://docs.vmware.com/en/vRealize-Operations/8.10/com.vmware.vcom.metrics.doc/GUID-41603CD6-453B-4E26-A237-34E733BAB00C.html).  #37257  #### Testing The metric was scraped from a test vCenter environment, and golden test files were updated to reflect the addition of the metric.  #### Documentation Documentation was updated according to the metadata.yaml  <img width="959" alt="Screenshot 2025-01-14 at 10 48 02 AM" src="https://github.com/user-attachments/assets/7c88902f-fe21-4c7d-be17-ba79a92222f3" /> <img width="919" alt="Screenshot 2025-01-15 at 11 35 31 AM" src="https://github.com/user-attachments/assets/77e19e19-93e4-410a-b501-6477c77e78da" /> Co-authored-by: Antoine Toulme <[email protected]>

#### Description The use and/or administration of geolocation databases might involve compiling with vendors licensing, this PR adds some words into the `geoIP` processor as a reminder.  #### Link to tracking issue Fixes  #### Testing  #### Documentation

``` internal/common/ttlmap/ttl_map.go:12:13: struct with 24 pointer bytes could be 16 internal/common/ttlmap/ttl_map.go:68:12: struct with 24 pointer bytes could be 16 ```

It doesn't appear disabling this linter is necessary anymore.

This comment says that the rule exclusions are only applying to test files, but that doesn't appear to be the case as no selection is made over file types. I offer to delete that comment as outdated.

#### Description Fixes goroutines leak by properly closing the underlying gRPC client which is only when we're using an insecure custom endpoint. Enables shutdown tests. #### Link to tracking issue Related to #30438 Co-authored-by: Kevin N. <[email protected]>

Signed-off-by: Antoine Toulme <[email protected]> Signed-off-by: Alex Boten <[email protected]> Co-authored-by: Alex Boten <[email protected]>

This is the same as was done in core in open-telemetry/opentelemetry-collector#12134 Signed-off-by: Alex Boten <[email protected]>

#### Description  Allowlists fuzzing targets

@mx-psi

#### Description An immediate fix for `make oteltestbedcol`. cc @mx-psi  #### Link to tracking issue Fixes #37627  #### Testing n/a  #### Documentation n/a

… in readme (#37696)  #### Description The Kafka receiver's readme is incorrect. TLS `insecure` should be `insecure_skip_verify`. See the following source code - https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/internal/kafka/authentication.go#L24 - https://github.com/open-telemetry/opentelemetry-collector/blob/main/config/configtls/configtls.go#L100

…37489)  #### Description This PR adds the following VM performance metrics for vcenter. 1. `vcenter.vm.cpu.time` 2. `vcenter.vm.network.multicast.packet.rate` 3. `vcenter.vm.network.broadcast.packet.rate` More information on these metrics can be found [here](https://docs.vmware.com/en/vRealize-Operations/8.10/com.vmware.vcom.metrics.doc/GUID-41603CD6-453B-4E26-A237-34E733BAB00C.html). and also succinctly described [here](https://www.servicenow.com/docs/bundle/xanadu-it-operations-management/page/product/agent-client-collector/reference/vsphere-metrics.html)  #37488  #### Testing The metrics were scraped from a test vCenter environment, and golden test files were updated accordingly to reflect the addition of the metric.  #### Documentation Documentation was updated according to the metadata.yaml <img width="1776" alt="evidence1" src="https://github.com/user-attachments/assets/662d6ff5-dda5-4e56-b124-5b401992871a" /> <img width="650" alt="Screenshot 2025-01-29 at 10 48 33 AM" src="https://github.com/user-attachments/assets/c421ed17-d718-4f78-b5d1-4b96627d7584" /> <img width="125" alt="Screenshot 2025-01-29 at 10 53 54 AM" src="https://github.com/user-attachments/assets/12317bf0-d767-4d6d-ad2f-0daf07c18555" /> <img width="933" alt="Screenshot 2025-01-29 at 10 54 11 AM" src="https://github.com/user-attachments/assets/331a31fb-0574-4465-8fc5-240be40e2684" /> Signed-off-by: Samiur Arif <[email protected]>

#### Description This PR addresses #36984 in order to open up telemetrygen to be used by golang tests and generate varying amounts of telemetry in code to support use cases which don't rely on external tools. The following major changes were made to clear the footprint for how this API can be used: 1. Moved `metrics`, `traces`, and `logs` under `pkg` instead of internal. 2. Unexported the `run` function for each test suite so the entrypoint is only the `Start` function 3. Moved config defaults from pflags into a `SetDefaults` function which is exported alongside `NewConfig` for changed packages in order for users to get the same sane defaults as if run from the command line. 4. Adjusted the pflags to use the existing default values rather than duplicate their own removing default config duplication 5. Added E2E tests for Metrics and Logs and adjusted Traces to use the new API so it stays consistent. 6. Adjusted log and metric exporter instantiation slightly due to a race condition with an `err` variable which was only caught by the new E2E tests. #### Testing Added to the E2E tests to ensure consistent API specs.

@dashpole

Fixes #37715 cc @dashpole @pjanotti Signed-off-by: Arthur Silva Sens <[email protected]>

Add code coverage of pod.container rule type.

#### Description Adds fuzz tests for receiver handlers and processing of metrics, logs and traces. For receivers, the fuzz test will create an HTTP request with the fuzzers testcase as its body. For processors, the fuzzer will create random logs, metrics or traces and pass it to a routine that processes the data - for the most part either a `processLogs`/`processMetrics`/`processTraces` method or `ConsumeLogs`/`ConsumeTraces` which will then invoke the internal `process` routine.  #### Link to tracking issue Fixes  #### Testing  #### Documentation  --------- Signed-off-by: Adam Korczynski <[email protected]> Co-authored-by: Pablo Baeyens <[email protected]>

…ied paths and error mode overrides (#37676)  #### Description This PR is part of #29017, and a split from #36888. It changes the `transformprocessor`, adding support for error mode overrides per context statements group, and global `conditions` with fully qualified paths for context-inferred configuration styles. Change log: - Structured context-inferred configuration styles (without `context: <value>`) now supports and requires conditions with fully qualified paths, for example: ```yaml log_statements: - statements: - set(log.attributes["pass"], "true") conditions: - log.attributes["pass"] == nil ``` If non qualified paths are used, it raises an error, for example `- attributes["pass"] == nil` would generate the following error: ``` invalid config for "transform" processor unable to parse OTTL condition "attributes[\"pass\"] == nil": missing context name for path "attributes[pass]", possibly valid options are: "log.attributes[pass]", "instrumentation_scope.attributes[pass]", "resource.attributes[pass]" ``` Existing non-context inferred configurations shouldn't be impacted by this change. - A new configuration `error_mode` key has been added to the context statements group (`common.ContextStatements`), When provided, it overrides the top-level error mode, offering more granular control over error handling, for example: ```yaml error_mode: propagate log_statements: - context: log error_mode: ignore statements: - set(attributes["attr1"], ParseJSON(1)) - context: log statements: - set(attributes["executed"], true) ```  #### Link to tracking issue #29017  #### Testing Unit tests  #### Documentation Changed the README to include the new configuration key `error_mode`  --------- Co-authored-by: Evan Bradley <[email protected]>

#### Description This is a second attempt at PR #37142, see that one for context. The PR was reverted in #37173, because the result of `make tidylist` was not a pure function of the repo state. The output of the tool depends on all `go.mod` files present in the repo directory, including those that may be `.gitignored`, such as the ones generated in `cmd/otelcontribcol` and `cmd/oteltestbedcol`. Since then, I [added a `--skip` flag to the tool](open-telemetry/opentelemetry-go-build-tools#662), which allows us to make the tool ignore these problematic `go.mod` files. Automatic detection of `.gitignored` files may be desirable if some developers have other non-gitted Go modules inside their repo directory. As this would require more advanced Git integration in the tool and should be easily avoided, I've decided to keep this more manual approach. #### Link to tracking issue Resolves #37336. See previous PR for impacted issues in core. #### Testing See previous PR for initial testing. I've tried running the tool locally, with and without the generated files in `cmd/otelcontribcol`, and the output was identical, confirming that the previous issue should be fixed. --------- Co-authored-by: Pablo Baeyens <[email protected]>

…ne exits (#37733) #### Description When WAL is used the shutdown doesn't wait until the go routine launched to handle WAL terminates. This makes the shutdown non-deterministic and cause the test failure issues linked below. Although a bug on the component it doesn't seem to deserve a changelog since it was detected via tests and there is no open issue around the shutdown of the component. #### Link to tracking issue Fixes #9124, #37715 #### Testing Ran the component tests locally multiple times on Windows (more likely to surface concurrency issues). #### Documentation N/A

…37723) Signed-off-by: Bogdan Drutu <[email protected]>

…names (#37741)  #### Description This PR adaps the `e2e` tests to run both the old syntax as it's, and the new one by leveraging the parser collection context prepend feature, giving us some extra coverage without having to duplicate every single test.  #### Link to tracking issue #29017

#### Description [Vulnerability scans](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/13184143287/job/36802088446?pr=37749) are detecting [GO-2025-3447](https://pkg.go.dev/vuln/GO-2025-3447) in our runs. This upgrades the test environment to run 1.22.12, the fixed version.

This PR updates the go-build-tools to the latest v0.18.0 version and adjusts to the breaking changes for the githubgen tool. [Link to the go-build-tools changelog](https://github.com/open-telemetry/opentelemetry-go-build-tools/blob/main/CHANGELOG.md#v0180) --------- Signed-off-by: Moritz Wiesinger <[email protected]>

Signed-off-by: Bogdan Drutu <[email protected]>

#### Description One option to help with open-telemetry/opentelemetry-collector#12311. We could also fix this after making that change, but I think the checks now better reflect the intent of the test.

OnConnectionClose was never hooked up to the server's OnConnectionClose, causing the restart of the agent to never be able to reconnect. --------- Signed-off-by: Alex Boten <[email protected]>

Before (including copy of the bytes needed upstream): ``` goos: darwin goarch: arm64 pkg: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer/internal/reader cpu: Apple M2 Max BenchmarkFileRead BenchmarkFileRead-12 6890 159408 ns/op 125802 B/op 132 allocs/op PASS ``` After ``` goos: darwin goarch: arm64 pkg: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer/internal/reader cpu: Apple M2 Max BenchmarkFileRead BenchmarkFileRead-12 7662 154923 ns/op 121656 B/op 130 allocs/op PASS ``` Signed-off-by: Bogdan Drutu <[email protected]>

#### Description Improvements to integration test: 1. Reduced test execution time by changing the behavior from starting and shutting down the database container for each test case to starting the container **once** and then running all tests for the given engine. Gain ~22s: [Main job](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/13077774968/job/36494091929): ``` DONE 47 tests, 8 skipped in 136.005s ``` [PR job](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/13077857640/job/36494335331?pr=37624) ``` DONE 44 tests, 4 skipped in 114.374s ``` 2. Updated the Oracle container from [gvenzl/oracle-xe:21-slim-faststart](https://hub.docker.com/r/gvenzl/oracle-xe) to [gvenzl/oracle-free:slim-faststart](https://hub.docker.com/r/gvenzl/oracle-free), primarily to support ARM architecture.  #### Testing * Ran tests for Oracle and SQL Server locally to ensure they are not broken. Co-authored-by: Curtis Robert <[email protected]>

@mx-psi

#### Description This PR fixed `make update-otel` to update `builder-config.yaml` correctly. This is the workflow we update `builder-config.yaml` files: 1. `MULTIMOD` command to update `go.mod` 2. `updatehelper` function to update `builder-config.yaml` according to the updated `go.mod` However, `make otelcontribcol/make oteltestbedcol` will overwrite the updated `go.mod` unexpectedly before step 2. This PR removes the unexpected `make otelcontribcol/make oteltestbedcol` commands. cc @mx-psi  #### Link to tracking issue n/a  #### Testing n/a  #### Documentation n/a

@atoulme

#### Description Proposing myself as a maintainer of this code. My membership has been sponsored on that purpose by @atoulme and @mx-psi open-telemetry/community#2508 ping also the current code owner @nslaughter  #### Link to tracking issue Fixes #37713  #### Testing  #### Documentation  Signed-off-by: Célian Garcia <[email protected]> Co-authored-by: Curtis Robert <[email protected]>

@MovieStoreGuy

#37749)  #### Description As component sponsor, @MovieStoreGuy [should be included as a code owner](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#adding-new-components).

#### Description Removes the previous assert.Equal statement that was causing the test to intermittently fail. The equal assertion was intended to be removed previously with the new logic but I missed it. Will keep an eye on this because I'm surprised I couldn't get it to happen locally (whereas before adding the new logic I could consistently get it to fail 2% of the time.) #### Link to tracking issue Fixes #34983

#### Description This PR adds a new gauge metric `datadog.otel.gateway`. This metric is set at `1` when the collector is used as a gateway otherwise, the value is `0`. If `datadog.host.name` was sent with two different values, then the collector is used as a gateway. Note: [The PR on opentelemetry-mapping-go](DataDog/opentelemetry-mapping-go#479) should be merged before this PR and `go.mod` must use a commit on main instead of `0.24.1-0.20250124131402-bcd6fd1617a6` Associated PRs: - DataDog/opentelemetry-mapping-go#479 - DataDog/datadog-agent#33370  #### Link to tracking issue Fixes  #### Testing The following commands generates the metric `datadog.otel.gateway` at `1` ``` telemetrygen metrics --otlp-insecure --otlp-attributes "datadog.host.name"=\"host1\" telemetrygen metrics --otlp-insecure --otlp-attributes "datadog.host.name"=\"host2\" ``` Same tests were performed for `traces` and for `logs` when `isLogsAgentExporterEnabled` is false.  #### Documentation  --------- Co-authored-by: Yang Song <[email protected]>

Signed-off-by: Bogdan Drutu <[email protected]>

#### Description This bash script precedes githubgen. It is now outdated and can be removed in favor of githubgen moving forward.  #### Link to tracking issue Fixes #37757

…authextension (#37334)  #### Description Adds an expiry_buffer config to the oauth2client extension, allowing token refresh before expiration. The default buffer is set to 5 m, preventing authentication failures due to token expiration.  #### Link to tracking issue Fixes #35148  #### Testing passes tests.  #### Documentation The README has been updated to include the expiry_buffer config option, with instructions on how to use it and its default value.

…` for the `numeric_attribute` (#37328)  #### Description This PR makes the `numeric_attribute` more flexible and allows to set only `min_value` or `max_value`, without the need to set both. This is useful to have simple configurations like these: ``` { type: numeric_attribute, numeric_attribute: { key: http.status_code, min_value: 400 } } ```  #### Link to tracking issue N/A  #### Testing Added unit tests to cover all changes and scenarios.  #### Documentation I fixed an example on the documentation to have just the `min_value` set

pull bot added the ⤵️ pull label Oct 18, 2024

codeboten requested a review from cparkins as a code owner October 18, 2024 18:41

axw and others added 28 commits January 16, 2025 15:48

[processor/logdedup] fix config validation not working (#37278)

10f52ed

#### Description The Config struct has a Validate method but it is only used in tests. Ensure we validate the configuration when creating the processor.

Upgrade otel core (#37279)

ae470e0

Signed-off-by: Bogdan Drutu <[email protected]>

[chore] Spelling exporter (#37133)

c506f54

#### Description Fix spelling in exporter/ #37128 (review) #### Link to tracking issue * #37128 --------- Signed-off-by: Josh Soref <[email protected]>

[chore] fix code owners (#37291)

fa1e76b

[chore] add atoulme as codeowner for jmxreceiver (#37296)

2acfccc

Proposing to take the position of codeowner of jmxreceiver. Moving this component out of unmaintained status.

[exporter/awss3] Implement sending queue (#37274)

20ecaff

 #### Description Implements the common queueing feature into the `awss3exporter`, as suggested [here](#36264).

Add JMX metrics gatherer version 1.43.0-alpha (#37302)

9cd3a36

Add JMX metrics gatherer version `1.43.0-alpha`. cc @open-telemetry/java-contrib-approvers

[chore] docs: fix nanosecond value range (#37309)

01104dd

[processor/tailsampling] Added @portertech to codeowners (#37299)

b28a329

[chore] apply fieldalignment to ttlmap (#37318)

1b93d01

``` internal/common/ttlmap/ttl_map.go:12:13: struct with 24 pointer bytes could be 16 internal/common/ttlmap/ttl_map.go:68:12: struct with 24 pointer bytes could be 16 ```

[receiver/awscloudwatchmetrics] mark unmaintained (#37323)

a204ecf

[chore] re-enable loopclosure, a noop with 1.22 (#37325)

3c1d010

It doesn't appear disabling this linter is necessary anymore.

[chore] remove outdated comment (#37327)

b026f73

This comment says that the rule exclusions are only applying to test files, but that doesn't appear to be the case as no selection is made over file types. I offer to delete that comment as outdated.

Update to latest core (#37344)

69d555d

Signed-off-by: Antoine Toulme <[email protected]> Signed-off-by: Alex Boten <[email protected]> Co-authored-by: Alex Boten <[email protected]>

[chore] apply fieldalignment (#37316)

b88759f

[chore] apply fieldalignment to s3 provider (#37315)

5a2f03a

[chore] apply gh client fix (#37354)

8f0c3e9

This is the same as was done in core in open-telemetry/opentelemetry-collector#12134 Signed-off-by: Alex Boten <[email protected]>

mx-psi and others added 30 commits February 5, 2025 18:59

Skip TestExportWithWALEnabled due to flakiness (#37719)

86430c5

Fixes #37715 cc @dashpole @pjanotti Signed-off-by: Arthur Silva Sens <[email protected]>

[chore] add pod.container test (#37634)

6638b83

Add code coverage of pod.container rule type.

Remove deprecated func BuildWithSplitFunc from stanza/fileconsumer (#…

d36ce1b

…37723) Signed-off-by: Bogdan Drutu <[email protected]>

[chore] Move test only code to a _test file (#37737)

2a42490

Signed-off-by: Bogdan Drutu <[email protected]>

[cmd/opampsupervisor] ensure OnConnectClosed is hooked up (#37761)

0820ea0

OnConnectionClose was never hooked up to the server's OnConnectionClose, causing the restart of the agent to never be able to reconnect. --------- Signed-off-by: Alex Boten <[email protected]>

[chore] Replace identical emittest.Call with emit.Token (#37765)

943ee27

Signed-off-by: Bogdan Drutu <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[pull] main from open-telemetry:main #47

[pull] main from open-telemetry:main #47

pull bot commented Oct 18, 2024 •

edited

Loading

[pull] main from open-telemetry:main #47

Are you sure you want to change the base?

[pull] main from open-telemetry:main #47

Conversation

pull bot commented Oct 18, 2024 • edited Loading

pull bot commented Oct 18, 2024 •

edited

Loading